ORTHOGONAL INVARIANCE AND IDENTIFIABILITY 

A. DANIILIDIS*, D. DRUSVYATSKIYt , AND A.S. LEWIS* 

Abstract. Orthogonally invariant functions of symmetric matrices often inherit properties from 
their diagonal restrictions: von Neumann's theorem on matrix norms is an early example. We 
discuss the example of "idcntifiability" , a common property of nonsmooth functions associated with 
r/*\ ' the existence of a smooth manifold of approximate critical points. Idcntifiability (or its synonym, 

"partial smoothness") is the key idea underlying active set methods in optimization. Polyhedral 
functions, in particular, are always partly smooth, and hence so are many standard examples from 
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1. Introduction. Nonsmoothncss is inherently present throughout even classi- 
r \ ' cal mathematics and engineering - the spectrum of a symmetric matrix variable is a 

good example. The nonsmooth behavior is not, however, typically pathological, but 
on the contrary is highly structured. The theory of identifiability (or its synonym, 
partial smoothness) |24[ 1201 1351 115] models this idea by positing existence of smooth 
manifolds capturing the full "activity" of the problem. Such manifolds, when they 
exist, are simply composed of approximate critical points of the minimized function. 
In the classical case of nonlinear programming, this theory reduces to the active set 
philosophy. Illustrating the ubiquity of the notion, the authors of [3] prove that iden- 
tifiable manifolds exist generically for convex semi- algebraic optimization problems. 
Identifiable manifolds arc particularly prevalent in the context of eigenvalue op- 
Q\ ' timization. One of our goals is to shed new light on this phenomenon. To this end, 

we will consider so-called spectral functions. These are functions F, defined on the 
space of symmetric matrices S™ , that depend on matrices only through their eigenval- 
ues, that is, functions that are invariant under the action of the orthogonal group by 
conjugation. Spectral functions can always be written as the composition F = f o X 
where / is a permutation-invariant function on R" and A is the mapping assigning to 
each matrix leS the vector of its eigenvalues (Xi(X), . . . , X n (X)) in non-increasing 
order, see [H Section 5.2]. Notable examples of functions fitting in this category are 
X H> Xi(X) and X >->• Y^i=i l-M-^OI- Though the spectral mapping A is very badly 
behaved, as far as say differentiability is concerned, the symmetry of / makes up for 
the fact, allowing powerful analytic results to become available. 

In particular, the Transfer Principle asserts that F inherits many geometric (more 
generally variational analytic) properties of /, or equivalently, F inherits many prop- 
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erties of its restriction to diagonal matrices. For example, when / is a permutation- 
invariant norm, then F is an orthogonally invariant norm on the space of symmet- 
ric matrices — a special case of von Neumann's theorem on unitarily invariant ma- 
trix norms |34j . The collection of properties known to satisfy this principle is im- 
pressive: convexity [23] [13] , prox- regularity [10] , Clarke-regularity [25] [23] , smooth- 
ness [23j[22l EH [Til [331 [32] , algebraicity [IT], and stratifiability [111 Theorem 4.8]. In 
this work, we add identifiability (and partial smoothness) to the list (Theorems 13. 171 
and 13.21]) . In particular, many common spectral functions (like the two examples 
above) can be written in the composite form Jo A, where / is a permutation-invariant 
convex polyhedral function. As a direct corollary of our results, we conclude that such 
functions always admit partly smooth structure! Furthermore, a "polyhedral-like" 
duality theory of partly smooth manifolds becomes available. 

One of our intermediary theorems is of particular interest. We will give an ele- 
mentary argument showing that a permutation-invariant set M is a C°° manifold if 
and only if the spectral set A~ 1 (M) is a C°° manifold (Theorem 12. 7p . The converse 
implication of our result is apparently new. On the other hand, the authors of [11] 
proved the forward implication even for C k manifolds (for k = 2, . . . , oo). This being 
said, their proof is rather long and dense, whereas the proof of our result is very 
accessible. The key idea of our approach is to consider the metric projection onto M. 

The outline of the manuscript is as follows. In Section [2] we establish some ba- 
sic notation and give an elementary proof of the spectral lifting property for C°° 
manifolds. In Section [3] we prove the lifting property for identifiable sets and partly 
smooth manifolds, while in Section@]we explore duality theory of partly smooth man- 
ifolds. Section [5] illustrates how our results have natural analogues in the world of 
nonsymmetric matrices. 

2. Spectral functions and lifts of manifolds. 

2.1. Notation. Throughout, the symbol E will denote a Euclidean space (by 
which we mean a finite-dimensional real inner-product space) . The functions that we 
will be considering will take their values in the extended real line R := RU {— oo, oo}. 
We say that an extended-real- valued function is proper if it is never {— oo} and is not 
always {+oo}. For a set QcE, the indicator function Sq : E — > R is a function that 
takes the value on Q and +oo outside of Q. An open ball of radius e around a 
point x will be denoted by B t (x), while the open unit ball will be denoted by B. Two 
particular realizations of E will be important for us, namely R™ and the space S™ of 
n x n-symmetric matrices. 

Throughout, we will fix an orthogonal basis of R™, along with an inner product 
(•,•). The corresponding norm will be written as || • ||. The group of permutations 
of coordinates of R" will be denoted by S n , while an application of a permutation 
a G S™ to a point x G R™ will simply be written as ax. We denote by R> the set 
of all points x G R ra with x\ > x-i > . . . > x n . A function /: R" — > R is said to be 
symmetric if we have f(x) — f(o~x) for every x G R" and every a G S". 

The vector space of real n x n symmetric matrices S n will always be endowed 
with the trace inner product (X, Y) = tr (-XT'), while the associated norm (Frobenius 
norm) will be denoted by | • 1 1 p . The group of orthogonal nxn matrices will be denoted 
by O™. Note that the group of permutations S" naturally embeds in O™. The action 
of O™ by conjugation on S" will be written as U.X :— U T XU, for matrices U G O™ 
and X G S™. A function h: S" — > R is said to be spectral if we have h{X) = h(U.X) 
for every X G S™ and every U G O™. 
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2.2. Spectral functions and the transfer principle. We can now consider 
the spectral mapping A : S™ — > R" which simply maps symmetric matrices to the 
vector of its eigenvalues in nonincreasing order. Then a function on S™ is spectral 
if and only if it can be written as a composition / o A, for some symmetric function 
/: R" — >• R. (See for example [23j Proposition 4].) As was mentioned in the introduc- 
tion, the Transfer Principle asserts that a number of variational-analytic properties 
hold for the spectral function / o A if and only if they hold for /. We will encounter 
a number of such properties in the current work. Evidently, analogous results hold 
even when / is only locally symmetric (to be defined below). The proofs follow by a 
reduction to the symmetric case by simple symmetrization arguments, and hence we 
will omit details in the current paper. 

For each point x £ R™, we consider the stabilizer 

Fix(x) := {a £ S" : ax = x}. 

Definition 2.1 (Local symmetry). A function /: R" — > R is locally symmetric 
at a point x £ R™ if we have f(x) = f{ax) for all points x near x and all permutations 
a £ Fix(x). 

A set Q C R" is symmetric (respectively locally symmetric) if the indicator 
function Sq is symmetric (respectively locally symmetric). The following shows that 
smoothness satisfies the Transfer Principle [32J |33] . 

Theorem 2.2 (Lifts of smoothness). Consider a function f: R™ — > R and a 
matrix X £ S™. Suppose that f is locally symmetric around x := X(X). Then f 
is C p -smooth (p = 1, . . . , oo) around x if and only if the spectral function / o A is 
C p -smooth around X . 

The distance of a point x to a set Q C E is simply 

d Q (x) :=inf {||x-j/|| : y £ Q}, 

whereas the metric projection of x onto Q is defined by 

P Q (x) :={y£Q: d Q (x) = y}. 

It will be important for us to relate properties of a set Q with those of the metric 
projection Pq. To this end, the following notion arises naturally [2"51l2l?] . 

Definition 2.3 (Prox-regularity) . A set Q C E is prox-regular at a point x £ 
Q if Q is locally closed around x and the projection mapping Pq is single-valued 
around x. 

In particular, all C 2 -manifolds and all closed convex sets are prox-regular around 
any of their points. See for example [3U1 Example 13.30, Proposition 13.32]. Addi- 
tionally, it is well-known that if M C E is a C p smooth manifold (for p > 2) around 
a point x £ M, then there exists a neighborhood U of x on which the projection Pm 
is single- valued and C p_1 -smooth. Prox-regularity also satisfies the transfer principle 
[TOl Proposition 2.3, Theorem 2.4]. 

Theorem 2.4 (Lifts of prox-regularity) . Consider a matrix X £ S n and a set 
Q C R" that is locally symmetric around the point x :— X(X). Then the function dQ 
is locally symmetric near x and the distance to the spectral set A _1 (Q) satisfies 

d\-i(Q) — dQ o A, locally around X. 
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Furthermore, Q is prox-regular at x if and only if A _1 (Q) is prox-regular at X . 
If a set Q C E is prox-regular at x, then the proximal normal cone 

N Q {x) := K+{v £~E:x£P Q {x + v)}, 

and the tangent cone 

Tq(x) :— < lim \{xi — x) : \ f oo and Xi £ Q >. 

are closed convex cones and are polar to each other [30l Corollary 6.29]. Here, we 
mean polarity in the standard sense of convex analysis, namely for any closed convex 
cone J(cE, the polar of K is another closed convex cone defined by 

K° := {v £ E : (v, w) < for all w £ K}. 

2.3. Lifts of symmetric manifolds. It turns out (not surprisingly) that smooth- 
ness of the projection Pq is inherently tied to smoothness of Q itself, which is the 
content of the following lemma. 

For any mapping F: E — > E, the directional derivative of F at x in direction w 
(if it exists) will be denoted by 

Db (x)(w) := lim , 

while the Gateaux derivative of F at x (if it exists) will be denoted by DF(x). 

Lemma 2.5 (Smoothness of the metric projection). Consider a set QcE that 
is prox-regular at a point x G Q. Then 

DP Q {x){v) = 0, for any v E N Q (x). (2.1) 

If Pq is directionally differentiate at x, then we also have 

DPq(x)(w) = w, for any w £T Q (x). (2.2) 

In particular, if Pq is Gateaux differ entiable atx, then Nq{x) and Tq(x) are orthog- 
onal subspaces and DPq(x) = Pt q (x)- If Pq is C k (k = l,...,ooj smooth near x, 
then Pq automatically has constant rank near x and consequently Q is a C k manifold 
around x. 

Proof. Observe that for any normal vector v 6 Nq(x) there exists e > so that 
Pq(x + e'v) = x for all nonnegative e' < e. Equation (|2.ip is now immediate. 

Suppose now that Pq is directionally differentiable at x and consider a vector 
w G Tq(x) with \\w\\ = 1. Then there exists a sequence Xi £ Q converging to x and 
satisfying w = limi^^ t^E^h- Define £j := \\Xi — x\\ and observe that since Pq is 
Lipschitz continuous, for some constant L we have 



\\PQ(x + tiW)- P Q (Xi)\\ <L 






ti 

and consequently this quantity converges to zero. We obtain 

nr, f-\t \ r Pq{x + Uw)-x P Q (xi)-x 
DPq(x){w) — hm — = lim -— 
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as claimed. 

Suppose now that Pq is Gateaux differentiable at x. Then clearly from (|2.1[) we 
have Nq(x) c ker DPq{x). If Nq(x) were a proper convex subset of ker D Pq(x), 
then we would deduce 

T Q {x) n ker DP Q {x) = [Nq(x)]° D ker£>P Q (x) / {0}, 

thereby contradicting equation (I2.2J) . Hence Nq(x) and Tq{x) are orthogonal sub- 
spaces and the equation DPq(x) — Pt„(x) readily follows from (|2.1|) and (I2.2[) . 

Suppose now that Pq is C fc -smooth (for k — 1. . . . , oo) around x. Then clearly 
we have 

rank D Pq (x) > rank DPq(x), for all x near x. 

Towards establishing equality above, we now claim that the set-valued mapping Tq 
is outer-semicontinuous at i. To see this, consider sequences Xi — > x and Wi <E 
TQ(xi), with ^ converging to some vector w € E. From equation (|2.2p . we deduce 
Wi = DPQ(xi)(wi). Passing to the limit, while taking into account the continuity of 
DPq, we obtain w = DPq(x)(w). On the other hand, since DPq(x) is simply the 
linear projection onto Tq(x), we deduce the inclusion w 6 Tq(x), and thereby estab- 
lishing outer-semicontinuity of Tq at 5. It immediately follows that the inequality, 
dim Tq (x) < dim Tq{x), holds for all x e Q near 5. 

One can easily verify that for any point x near x, the inclusion Nq(Pq(x)) C 
ker DPq (x) holds. Consequently we deduce 

rank DP Q (x) < dim Tq(Pq (x)) < dimT Q (x) = rank DPq (x), 

for all x £ E sufficiently close to x. as claimed. Hence Pq has constant rank near x. 
By the constant rank theorem, for all sufficiently small e > 0, the set PQ(B e (x)) is a 
C fe manifold. Observing that the set Pq(B c {x)) coincides with Q near x completes 
the proof. □ 

The following observation will be key. It shows that the metric projection map 
onto a prox-regular set is itself a gradient of a C 1 -smooth function. This easily follows 
from 29, Proposition 3.1]. In the convex case, this observation has been recorded and 
used explicitly for example in [181 Proposition 2.2] and [211 Preliminaries], and even 
earlier in [2] and |36) . 

Lemma 2.6 (Projection as a derivative). Consider a set Q C E that is prox- 
regular at x. Then the function 

h{x):=\\\x\\ 2 ~ l -dl{x), 

is ^-smooth on a neighborhood of x, with Vh(x) — Pq(x) for all x near x. 
We are now ready to state and prove the main result of this section. 

Theorem 2.7 (Spectral lifts of manifolds). Consider a matrix X € S™ and a 
set M C R™ that is locally symmetric around x := A(X). Then M is a C°° manifold 
around x if and only if the spectral set \^ 1 (M) is a C°° manifold around X. 

Proof. Consider the function 

Kx):=\\\xf-\^ M {x). 
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Suppose that M is a C°° manifold around x. In particular M is prox-regular, see 
[501 Example 13.30]. Then using Theorem 12.41 we deduce that h is locally symmetric 
around x. In turn, Lemma 12.61 implies the equality V/i = Pm near S. Since M is 
a C°° manifold, the projection mapping Pm is C°°-smooth near x. Combining this 
with Theorem l2.2[ we deduce that the spectral function h o A is C°°-smooth near X. 
Observe 

(ho\)(X) = ±\\\(X)f-±d 2 M (\(X)) 

= \\m\ 2 F -\d\- H M){x), 

where the latter equality follows from Thcorcm l2.4l Applying Theorcm l2.4[ we deduce 
that X~ 1 (M) is prox-regular at X. Combining this with Lemma l2~6l we obtain equality 
V(/i o X)(X) = P\-i(m)(X) f° r au % near X. Consequently the mapping X n- 
P\-i(M)(X) IS C°°-smooth near X. Appealing to Lemma 12.51 we conclude that 
X~ 1 (M) is a C°° manifold. The proof of the converse implication is analogous. D 

Remark 2.8. The proof of Theorem 12.71 falls short of establishing the lifting 
property for C manifolds, with k is finite, but not by much. The reason for that 
is that C fc manifolds yield projections that are only C fe ~ 1 smooth. Nevertheless, the 
same proof shows that C k manifolds do lift to C k ^ 1 manifolds, and conversely C k 
manifolds project down by A to C fc_1 manifolds. 

2.4. Dimension of the lifted manifold. The proof of Thcorcm l2.7l is relatively 
simple and short, unlike the involved proof of [11] , One shortcoming however is that it 
does not a priori yield information about the dimension of the lifted manifold A -1 (Af ) . 
In this section, we outline how we can use the fact that A _1 (M) is a manifold to 
establish a formula between the dimensions of M and A _1 (M). This section can 
safely be skipped upon first reading. 

We adhere closely to the notation and some of the combinatorial arguments of 
[XT] and [?]. With any point x £ R™ we associate a partition V x — {h, ■ ■ ■ , Ip} of the 
set {1, . . . , n}, whose elements are defined as follows: 

i,j e h ^=^ Xi = xj. 
It follows readily that for x 6 R™ there exists a sequence 

1 = io < i\ < . . . < i p = n 
such that 

h = {U-i, ■■■, it], for each ie{l,...,p}. 
For any such partition V we set 

A v := {x € R| : V x = V}. 

As explained in [Til Section 2.2], the set of all such Aj>'s defines an affine stratification 
of R™ . Observe further that for every point x € R> we have 

\- l ( x ) = {U T XU : U G O"}. 
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Let O n x := {U e O" : U T XU = X} denote the stabilizer of X, which is a C c 
manifold of dimension 



dimO^ = di 



as one can easily check. Since the orbit A _1 (:r) is isomorphic to 0"/0^, it follows 
that it is a submanifold of S". A computation, which can be found in [TT], then yields 
the equation 

dim \-\x) = dim O" - dim O n x = ^ \*i\ \ T J I ■ 

l<i<j<p 

Consider now any locally symmetric manifold M of dimension d. There is no loss 
of generality to assume that M is connected and has nonempty intersection with R" . 
Let us further denote by A* an affine stratum of the aforementioned stratification 
of R" with the property that its dimension is maximal among all of the strata A 
enjoying a nonempty intersection with M. It follows that there exists a point x S 
MnA, and S > satisfying MnB(x,8) C A* (see [TTJ Section 3] for details). Since 
dim A _1 (M) = dim A _1 (M n B(x,S)) and since X' 1 (M n B{x,8)) is a fibration we 
obtain 

dim A _1 (M) = dim M + ^ \I?\ |Jt|, (2.3) 

i<j<i<p» 

where V* — {/*,...,/*} is the partition associated to x (or equivalently, to any 
x e A*). 

Remark 2.9. It's worth to point out that it is possible to have strata Ai ^ A2 
of R™ of the same dimension, but giving rise to stabilizers of different dimension 
for their elements. The argument above shows that a connected locally symmetric 
manifold cannot intersect simultaneously these strata. This also follows implicitly 
from the forthcoming Lemma 14.41 asserting the connectedness of A _1 (Af), whenever 
M is connected. 

3. Spectral lifts of identifiable sets and partly smooth manifolds. We 

begin this section by summarizing some of the basic tools used in variational analysis 
and nonsmooth optimization. We refer the reader to the monographs of Borwein-Zhu 
[5], Clarke-Ledyaev-Stern-Wolenski [9 , Mordukhovich 26 ], and Rockafellar-Wets [3D] 
for more details. Unless otherwise stated, we follow the terminology and notation of 

m> 

3.1. Variational analysis of spectral functions. For a function /: E — >• R, 
the domain of / is 

dom/ := {x e E : f(x) < +00}, 

and the epigraph of / is 

cpi/:={(.i,r)eExR: r >/(4 

We will say that / is lower semicontinuous (Isc for short) at a point x provided that the 
inequality liminf^-j.^ f{x) > f{x) holds. If / is lower semicontinuous at every point, 
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then we will simply say that / is lower semicontinuous. For any set Q, the symbols 
clQ, convQ, and aSQ will denote the topological closure, the convex hull, and the 
affine span of Q respectively. The symbol par Q will denote the parallel subspace of 
Q, namely the set parQ := (aff Q) — Q- For convex sets QgE, the symbols riQ and 
rb Q will denote the relative interior and the relative boundary of Q, respectively. 

Given any set Q C E and a mapping /: Q — > Q, where Q is a subset of some 
other Euclidean space F, we say that / is C p - smooth if for each point ieQ, there is 
a neighborhood U of x and a C p -smooth mapping / : E — > F that agrees with / on 

Qnu. 

Recall that by Theorem l2.2[ smoothness of functions satisfies the Transfer Princi- 
ple. Shortly, we will need a slightly strengthened version of this result, where smooth- 
ness is considered only relative to a certain locally symmetric subset. We record it 
now. 

Corollary 3.1 (Lifts of restricted smoothness). Consider a function /: R™ — > 
R, a matrix X £ S™, and a set M C R" containing x := X(X). Suppose that f 
and M are locally symmetric around x. Then the restriction of f to M is C p -smooth 
(p = 1, . . . , ooj around x if and only if the restriction of fo A to A _1 (Af ) is C p -smooth 
around X. 

Proof. Suppose that the restriction of / to M is C p -smooth around x. Then 
there exists a C p -smooth function /, defined on R", and agreeing with f on M near 
x. Consider then the symmetrized function 



1 

ill 

aeFix(s) 



^ m(x) - TpwW E K ax ^ 



where |Fix(x)| denotes the cardinality of the set Fix(x). Clearly / sym is C p -smooth, 
locally symmetric around x, and moreover it agrees with / on M near x. Finally, using 
Theorem 12. 2\ we deduce that the spectral function / sym o A is C p -smooth around X 
and it agrees with / o A on A _1 (M) near X. This proves the forward implication of 
the corollary. 

To see the converse, define F := /°A, and suppose that the restriction of F to A _1 (M) 
is C p -smooth around X. Then there exists a C p -smooth function F, defined on S™, 
and agreeing with F on X^ 1 (M) near X. Consider then the function 



F ^( X ) :=^ F(U.X), 



C/GO™ 

where |O n | denotes the cardinality of the set O™. Clearly F sym is C p -smooth, spectral, 
and it agrees with F on A~ 1 (M) near X. Since F sym is spectral, we deduce that there 
is a symmetric function / on R" satisfying F sym = / o A. Theorem 12.21 then implies 
that / is C p -smooth. Hence to complete the proof, all we have to do is verify that 
/ agrees with f on M near x. To this end consider a point x £ M near x and 
choose a permutation a G Fix(x) satisfying ax 6 R™. Let U € O" be such that 
X = U T (Dia,gx)U. Then we have 

/» = f(ax) - F sym (C/ T (Dia ga; )[/) - F(U T (Diagx)U) - f(ax) = f(x), 

as claimed. □ 

Subdifferentials are the primary variation- analytic tools for studying general non- 
smooth functions / on E. 
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Definition 3.2 (Subdifferentials). Consider a function /: E — >• R and a point 
x with f{x) finite. 

(i) The Frechet subdifferential of / at x, denoted df(x), consists of all vectors 
»£E satisfying 

f(x) > f(x) + (v,x -x) + o(||a;-as||). 

(ii) The limiting subdifferential of / at x, denoted df(x), consists of all vec- 
tors v £ E for which there exist sequences Xi £ E and Vi £ df{xi) with 
(xi,f(xi),Vi) -)■ (x,f(x),v). 
Let us now recall from 23 , Proposition 2] the following lemma, which shows that 
subdifferentials behave as one would expect in presence of symmetry. 

Lemma 3.3 (Subdifferentials under symmetry). Consider a function f : R" — ¥ R 
that is locally symmetric at x. Then the equation 

df(ax) = adf{x), holds for any a £ Fix(x) and all x near x. 

Similarly, in terms of the spectral function F := / o A, we have 

dF(U.X) = U.(dF(X)), for any U £ O". 

Remark 3.4. In particular, if /: R™ — >• R is locally symmetric around x, then 
the sets df(x), ridf(x), rbdf(x), &ffdf(x), and par<9/(a;) are invariant under the 
action of the group Fix($). 

The following result is the cornerstone for the variational theory of spectral map- 
pings [23l Theorem 6]. 

Theorem 3.5 (Subdifferential under local symmetry). Consider a Isc function 
f : R rl — > R and a symmetric matrix l£S™, and suppose that f is locally symmetric 
at X(X). Then we have 

d(f o X)(X) = {C/ T (Diagw)C7 : v £ df(X(X)) and U £ O^}, 

where 

O n x = {[/eO":I = C/ T (DiagA(X))t/}. 

It is often useful to require a certain uniformity of the subgradients of the function. 
This is the content of the following definition 28, Definition 1.1]. 

Definition 3.6 (Directional prox-regularity) . A function /: E — » R is called 
prox-regular at x for v, with v £ df(x), if / is locally lsc at x and there exist e > 
and p > so that the inequality 

f(y)>f(x) + (v,y-x)-^\\ y -x\\ 2 , 

holds whenever x,y £ B e (x), v £ B e (v) fl df(x), and f(x) < f{x) + e. 

The function / is called prox-regular at x, if it is finite at x and / is prox-regular at 

x for every subgradient v £ df(x). 

A set Q C R™ is prox-regular at x for v £ Nq(x) provided that the indicator function 

Sq is prox-regular at x for v. 
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In particular C 2 -smooth functions and lsc, convex functions are prox-regular at 
each of their points [30, Example 13.30, Proposition 13.34]. 

Remark 3.7. A set Q C E is prox-regular at x, in the sense above, if and only 
if it is prox-regular in the sense of Definition 12.31 For a proof, see for example [301 
Exercise 13.38]. 

The following theorem shows that directional prox-regularity also satisfies the 
Transfer Principle [TU1 Theorem 4.2]. 

Theorem 3.8 (Directional prox-regularity under spectral lifts). 
Consider a lsc function /: R n — > R and a symmetric matrix X. Suppose that f is 
locally symmetric around x := X(X). Then f is prox-regular at x if and only if f o A 
is prox-regular at X . 

The following two standard results of Linear Algebra will be important for us [331 
Proposition 3]. 

Lemma 3.9 (Simultaneous Conjugacy). Consider vectors x,y,u,v £ R™. Then 
there exists an orthogonal matrix U £ O n with 

Diagrr = U T (Dia,gu)U and Diagy = C/ T (Diag v)U, 

if and only if there exists a permutation a € S n with x = au and y = av. 

Corollary 3.10 (Conjugations and permutations). Consider vectors Vi,V2 G 
R n and a matrix X £ S". Suppose that for some U\, U2 £ O^- we have 

J7f(Diag«i)J7i = C/ 2 T (Diag« 2 )[/ 2 . 

Then there exists a permutation a G Fix(A(A)) satisfying o~v\ — V2- 
Proof. Observe 

(C/ 1 C/ 2 T ) T Diag Vl (U! Ul) = Diag« 2 , 



{UiUi Y DiagA(X)(f/iC/ 2 i ) = Diag \{X). 

The result follows by an application of Lemma 13.91 D 

3.2. Main results. In this section, we consider partly-smooth sets, introduced 
in [24]. This notion generalizes the idea of active manifolds of classical nonlinear 
programming to an entirely representation-independent setting. 

Definition 3.11 (Partial Smoothness). Consider a function /: E — > R and a 
set M C E containing a point x. Then / is C p -partly smooth (p = 2, . . . , 00) at x 
relative to M if 

(i) (Smoothness) M is a C p manifold around x and / restricted to M is C p - 
smooth near x, 

(ii) (Regularity) / is prox-regular at x, 

(hi) (Sharpness) the affine span of df is a translate of Nm{x), 

(iv) (Continuity) df restricted to M is continuous at x. 
If the above properties hold, then we will refer to M as the partly smooth manifold of 
/ at x. 

Remark 3.12. Though the original definition of partial smoothness replaces 
the prox-regularity condition by Clarke-regularity, we feel that the prox-regularity is 
essential for the theory. In particular, without it, partly-smooth manifolds are not 
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even guaranteed to be locally unique and the basic property of identifiability may fail 
HOI Section 7]. 

Some comments are in order. First the continuity property of df is meant in the 
Painleve-Kuratowski sense. See for example 30, Definition 5.4]. The exact details of 
this notion will not be needed in our work, and hence we do not dwell on it further. 
Geometrically, partly smooth manifolds have a characteristic property in that the 
epigraph of / looks "valley-like" along the graph of f\ M . See Figure EH] for an 
illustration. 




Figure 3.1. The partly smooth manifold M for f(x, y) := \x\(l — \x\) + y 2 . 



It is reassuring to know that partly smooth manifolds are locally unique. This is 
the content of the following theorem J20J Corollary 4.2]. 

Theorem 3.13 (Local uniqueness of partly smooth manifolds) . Consider a func- 
tion f: E — > R that is C p -partly smooth (p > 2) at x relative to two manifolds M\ 
and Mi. Then there exists a neighborhood U of x satisfying U l~l M% = U H Ma. 

Our goal in this section is to prove that partly smooth manifolds satisfy the Trans- 
fer Principle. However, proving this directly is rather difficult. This is in large part 
because the continuity of the subdifferential mapping d(foX) seems to be intrinsically 
tied to continuity properties of the mapping 



X^O n x 



{[/GO":I = £/ T (DiagA(X))[/}, 



which are rather difficult to understand. 

We however will side-step this problem entirely by instead focusing on a property 
that is seemingly different from partial smoothness — finite identification. This notion 
is of significant independent interest. It has been implicitly considered by a number of 
authors in connection with the possibility to accelerate various first-order numerical 
methods [35] QH HZ] [TJ [02 CE2 , and has explicitly been studied in [15] for its 
own sake. 

Definition 3.14 (Identifiable sets). Consider a function /: E — >• R, a point 
x £ R™, and a subgradicnt v 6 df{x). A set M C dom/ is identifiable at x for v if 
for any sequences (xi, f{xi),Vi) — > (x,f(x),v), with w,; e df(xi), the points Xi must 
all lie in M for all sufficiently large indices i. 

Remark 3.15. It is important to note that identifiable sets are not required to 
be smooth manifolds. Indeed, as we will see shortly, identifiability is a more basic 
notion than partial smoothness. 
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The relationship between partial smoothness and finite identification is easy to 
explain. Indeed, as the following theorem shows, partial smoothness is in a sense just 
a "uniform" version of identifiability [THl Proposition 9.4]. 

Proposition 3.16 (Partial smoothness and identifiability). Consider a Isc func- 
tion f : E — >• R that is prox-regular at a point x. Let M C dom/ be a C p manifold 
(p = 2, . . . , ooj containing x, with the restriction f\ M being C p -smooth near x. Then 
the following are equivalent 

1. f is G p -partly smooth at x relative to M 

2. M is an identifiable set (relative to f ) at x for every subgradient v G v'idf(x). 

In light of the theorem above, our strategy for proving the Transfer Principle for 
partly smooth sets is two-fold: first prove the analogous result for identifiable sets 
and then gain a better understanding of the relationship between the sets ri df{\{X)) 
andri0(/oA)(X). 

Proposition 3.17 (Spectral lifts of Identifiable sets). Consider a Isc function 
f: R" — > R and a symmetric matrix X £ S™. Suppose that f is locally symmetric 
around x :— X(X) and consider a subset M C R™ that is locally symmetric around x. 
Then M is identifiable (relative to f) at x for v £ df{x), if and only if X~ 1 (M) is 
identifiable (relative to f o X) at X for U T (Di&gv)U £ d(f o X)(X), where U £ O^ is 
arbitrary. 

Proof. We first prove the forward implication. Fix a subgradient 

V := Z7 T (Diag v)U G d(f o X)(X), 

for an arbitrary transformation U £ O^. For convenience, let F :— /o A and consider 

a sequence (Xi,F(Xi), Vi) — > (X,F(X),V). Our goal is to show that for all large 
indices i, the inclusion A(Xj) G M holds. To this end, there exist matrices Ui G O^-. 
and subgradients Vi £ 9/(A(X,)) with 

Uf (DiagA(X J ))C/ l = Xi and Uj (Diag^U* = V t . 

Restricting to a subsequence, we may assume that there exists a matrix U £ O^ sat- 
isfying Ui — >• U, and consequently there exists a subgradient v £ df(X(X)) satisfying 
Vi — > v. Hence we obtain 

U T (Di&gX(X))U = X and U T (Dizgv)U = V = Tj T (Dmgv)U. 

By Corollary 13.101 there exists a permutation a £ Fix(a;) with crv = v. Observe 
(X(Xi), f(X(Xi)),Vi) — >• (x,f(x),v). Observe that the set a~ l M is identifiable (rela- 
tive to /) at x for v. Consequently for all large indices i, the inclusion A(X^) G o~ x M 
holds. Since M is locally symmetric at x, we deduce that all the points A(AQ) even- 
tually lie in M . 

To see the reverse implication, fix an orthogonal matrix U £ Oj and define V := 

U (Diag-D)C/. Consider a sequence (xj, f(Xi),Vi) — > (x, f(x),v) with Vi £ df(xi). 
It is not difficult to see then that there exist permutations <7; £ Fix(x) satisfying 
o~iXi £ R>. Restricting to a subsequence, we may suppose that o~i are equal to a fixed 
a £ Fix(S). Define 

Xi := U (Diagaxi)U and Vj := U (Dia,gavi)U. 
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Letting A a -\ G O" denote the matrix representing the permutation cr -1 , we have 

X t := {U T A a -iU) T [U T (DiagXi)U]U T A a -iU and 

V % := (U T A a -iU) T [U T {Diagv z )U]U T A a -iU. 

We deduce X t -> (TJ T A a -tU) T X(jJ T A a -iU) and V % -> (JJ T A a -iU) T V{U T A a -iU). 
On the other hand, observe X = (£7 T ,4 CT -it7) T A([7 T ,4 CT -it7). Since A _1 (M) is identi- 

rp rp 

fiable (relative to F) at X for (77 A a -iU) T V(U A„-iU), we deduce that the matri- 
ces Xi lie in A~ 1 (M) for all sufficiently large indices i. Since M is locally symmetric 
around x, the proof is complete. D 

Using the results of Section [2j we can now describe in a natural way the affinc 
span, relative interior, and relative boundary of the Frechet subdifferential. We begin 
with a lemma. 

Lemma 3.18 (Affine generation). Consider a matrix X G S™ and suppose that 
the point x := X(X) lies in an affine subspace V C R ra that is invariant under the 
action of Fix(a>) . Then the set 

{U T (Di&gv)U :veV andU G O^}, 

is an affine subspace of S n . 

Proof. Define the set L := (par V) ± . Observe that the set LflV consists of a 
single vector; call this vector w. Since both L and V are invariant under the action 
of Fix(x), we deduce aw = w for all a £ Fix(jc). 

Now define a function g : R n — >• R by declaring 

g(y) = (w,y)+S x+L (y), 

and note that the equation 

dg(x) := w + N x+ l(x) = V, holds. 

Observe that for any permutation a G Fix(x), we have 

g(ay) = {w,cry} + 5 X+L {ay) = (o-~ 1 w,y) +S x+IJ -i L (y) = g(y). 

Consequently g is locally symmetric at x. Observe 

(goX)(Y) = (w,X(Y)} + S x - l{x+L) Y. 

It is immediate from Theorems 12.21 and 12.71 that the function Y H> (w,X(Y)) is C°°- 
smooth around X and that X~ l {x + L) is a C°° manifold around X. Consequently 
d(g o X)(X) is an affinc subspace of S™. On the other hand, we have 

d(g o X)(X) = {U T (Bmgv)U : v G V and U G O^}, 

thereby completing the proof. D 

Proposition 3.19 (Affine span of the spectral Frechet subdifferential) . Consider 
a function f: R" — > R and a matrix X G S". Suppose that f is locally symmetric at 
X(X). Then we have 

&Sd{f o X)(X) = {U T (Bi&gv)U : v G aff df{X(X)) and U G O^}, (3.1) 

rb<9(/ o X)(X) = {U T (Bi&gv)U : v G rbdf(X(X)) and U G O^}. (3.2) 



nd(f o X)(X) = {U 1 (Diagu)t/ : v G ridf(X(X)) and U G O^}. (3.3) 
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Proof. Throughout the proof, let x := X(X). We prove the formulas in the order 
that they are stated. To this end, observe that the inclusion D in (13. ip is immediate. 
Furthermore, the inclusion 

8{f o \){X) C {U T (Biagv)U : v G &Sdf{\(X)) and U G O n x ). 

clearly holds. Hence to establish the reverse inclusion in (j3.ll) . it is sufficient to show 
that the set 

{U T (Diagv)U : v G &ftdf(X(X)) and U G O™ }, 

is an affine subspace; but this is immediate from Remark l3.4l and Lemma 13.181 Hence 
(|33|) holds. 

We now prove dSSJ). Consider a matrix U T (Diagv)U G rb<9(/ o X)(X) with U e 0\ 
and v G df(X(X)). Our goal is to show the stronger inclusion v G rb<9/(x). Observe 
from (|3.1[) . there exists a sequence £/j T (DiagVi)[/; — ► C/ T (Diagw)C/ with Ui G O^-, 
«i G aff9/(a;), and v^ ^ df(x). Restricting to a subsequence, we may assume that 
there exists a matrix U G O^- with Ui — >• [7 and a vector u G aff df(x) with u^ — > v. 
Hence the equation 

U T (Dmgv)U = U T (Bisigv)U, holds. 

Consequently, by Corollary 13.101 there exists a permutation a G Fix(x) satisfying 
ov = v. Since df{x) is invariant under the action of Fix(cc), it follows that v lies in 
rb9/(a;), and consequently from Rcmark l3.4l we deduce v G rbdf(x). This establishes 
the inclusion C of (|3.2[) . To see the reverse inclusion, consider a sequence Vi G aff df(x) 
converging to v G df(x) with Vi ^ df(x) for each index i. Fix an arbitrary matrix 
U G O^- and observe that the matrices C/ T (Diag Vi)U lie in aff d(fo\)(x) and converge 
to U T (Diagv)U. We now claim that the matrices U T (Di&gVi)U all lie outside of 
d(f o A) (a;). Indeed suppose this is not the case. Then there exist matrices Ui G O^- 
and subgradients Vi G df(x) satisfying 

U T {Vmg Vl )U = ^(Diagw,)^- 

An application of Corollary 13.101 and Remark 13.41 then yields a contradiction. There- 
fore the inclusion C/ T (Diagw)[7 G rbd(foX){X) holds, and the validity of (J3.2I) follows. 
Finally, we aim to prove (I3.3[) . Observe that the inclusion C of (|3.3[) is immediate 
from equation (|3.2|) . To see the reverse inclusion, consider a matrix f/ T (Diagw)C/, for 
some U G O^- and v G ridf(x). Again, an easy application of Corollary 13.101 and 
Remark |3 . 41 yields the inclusion U T (Di&gv)U G ri<9(/oA)(A). We conclude that (13.31) 
holds. D 

Lemma 3.20 (Symmetry of partly smooth manifolds). Consider a Isc function 
f: R" —5- R that is locally symmetric at x. Suppose that f is C p -partly smooth at x 
relative to M . Then M is locally symmetric around x. 

Proof. Consider a permutation a G Fix(a;). Then the function / is partly smooth 
at x relative to aM . On the other hand, partly smooth manifolds are locally unique 
Theorem 13.131 Consequently we deduce equality M = aM locally around x. The 
claim follows. D 

The main result of this section is now immediate. 
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Theorem 3.21 (Lifts of C°°-partly smooth functions). Consider a Isc function 
f : R" — > R and a matrix X G S™ . Suppose that f is locally symmetric around 
x := X(X). Then f is C°° -partly smooth at x relative to M if and only if f o A is 
C°° -partly smooth at X relative to X~ 1 (M). 

Proof. Suppose that / is C°°-partly smooth at x relative to M. In light of 
Lemma GOQl we deduce that M is locally symmetric at x. Consequently, Theorem l2.7l 
implies that the set X~ 1 (M) is a C°° manifold, while Corollary 13.11 implies that / o A 
is C°°-smooth on A _1 (M) near X. Applying Theorem 13.81 we conclude that /oA 
is prox-regular at X. Consider now a subgradient V G ri<9(/ o X)(X). Then by 
Proposition [3T9l there exists a vector v G ridf(x) and a matrix U G O^ satisfying 

V = U T (T)[agv)U and X = U T (T)iagx)U. 

Observe by Proposition 13.161 the set M is identifiable at x for v. Then applying 
Proposition l3.171 we deduce that X~ 1 (M) is identifiable (relative to /oA) at X relative 
to V. Since V is an arbitrary element of rid(f o A)(X), applying Proposition 13.161 
we deduce that / o A is C°°-partly smooth at X relative to X~ 1 (M), as claimed. The 
converse follows along the same lines. D 

The forward implication of Theorem 13.211 holds in the case of C p -partly smooth 
functions (for p = 2,...,oo). The proof is identical except one needs to use [111 
Theorem 4.21] instead of Theorem 12.71 We record this result for ease of reference in 
future works. 

Theorem 3.22 (Lifts of C p -partly smooth functions). Consider a Isc function 
f : R" — > R and a matrix X G S™ . Suppose that f is locally symmetric around 
x := X(X) . If f is C p -partly smooth (for p = 2, . . . , ooj at x relative to M , then / o A 
is C°° -partly smooth at X relative to X~ l {M). 

4. Partly smooth duality for polyhedrally generated spectral functions. 

Consider a Isc, convex function / : E — >• R. Then the Fenchel conjugate f* : E — ► R 
is defined by setting 

f*{y) = sup {{x,y} - f(x)}. 

ieR" 

Moreover, in terms of the powerset of E, denoted P(E), we define a correspondence 
J f : P(E) -^ P(E) by setting 

J f (Q):= \Jridf(x). 
xeQ 

The significance of this map will become apparent shortly. Before proceeding, we 
recall some basic properties of the conjugation operation: 

Biconjugation: /** = /, 

Subgradient inversion formula: df* = (df) _1 , 

Fenchel- Young inequality: (x,y) < f(x) + f*(y) for every x,y G R™. 

Moreover, convexity and conjugation behave well under spectral lifts. See for 
example [H Section 5.2]. 

Theorem 4.1 (Lifts of convex sets and conjugation). If f: R" — > R is a sym- 
metric function, then f* is also symmetric and the formula 

(f o A)* = /* o A, holds. 
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Furthermore f is convex if and only if the spectral function f o X is convex. 

The following definition is standard. 

Definition 4.2 (Stratification). A finite partition A of a set Q c E is a strat- 
ification provided that for any partitioning sets (called strata) M% and M% in A, the 
implication 

Mi n clM 2 ^ =4- Mi C clM 2 , holds. 

If the strata are open polyhcdra, then .A is a polyhedral stratification. If the strata 
are C fe manifolds, then A is a C k - stratification. 

Stratification duality for convex polyhedral functions. We now establish 
the setting and notation for the rest of the section. Suppose that / : R" — > R is 
a convex polyhedral function (epigraph of / is a closed convex polyhedron). Then 
/ induces a finite polyhedral stratification Af of dom / in a natural way. Namely, 
consider the partition of epi/ into open faces {Fi}. Projecting all faces Fi, with 
dim Fi < n, onto the first n-coordinates we obtain a stratification of the domain 
dom/ of / that we denote by Af. In fact, one can easily see that / is C°°-partly 
smooth relative to each polyhedron M € Af. 

A key observation for us will be that the correspondence / <s— > f* is not only 
a pairing of functions, but it also induces a duality pairing between Af and Af* . 
Namely, one can easily check that the mapping Jf restricts to an invertible mapping 
Jf : Af — > Af* with inverse given by Jf* . 

Limitations of stratification duality. It is natural to ask whether for general 
(nonpolyhedral) lsc, convex functions /: R™ — > R, the correspondence / •<-» /*, along 
with the mapping J, induces a pairing between partly smooth manifolds of / and 
/*. Little thought, however shows an immediate obstruction: images of C°°-smooth 
manifolds under the map Jf may fail to be even C 2 -smooth. 

Example 4.3 (Failure of smoothness). Consider the conjugate pair 

1 o 

f(x,y) = -^(x 4 +y 4 ) and f*(x,y) = -(|x|f + \y\§). 

Clearly / is partly smooth relative to R 2 , whereas any possible partition of R 2 into 
partly smooth manifolds relative to /* must consist of at least three manifolds (one 
manifold in each dimension: one, two, and three). Hence no duality pairing between 
partly smooth manifolds is possible. See the Figures |4~T1 and |4~21 for an illustration. 

Indeed, this is not very surprising, since the convex duality is really a duality 
between smoothness and strict convexity. See for example [27j Section 4] or [30] The- 
orem 11.13]. Hence in general, one needs to impose tough strict convexity conditions 
in order to hope for this type of duality to hold. Rather than doing so, and more in 
line with the current work, we consider the spectral setting. Namely, we will show 
that in the case of spectral functions F := / o A, with / symmetric and polyhedral 
— functions of utmost importance in eigenvalue optimization — the mapping J does 
induce a duality correspondence between partly smooth manifolds of F and F* . 

In the sequel, let us denote by 



M sym := (J o-M 
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Figure 4.1. {(x, y) : x 4 + j/ 4 < 4} 




Figure 4.2. {(a:,j/) : \x\s + \y\s < |} 



the symmetrization of any subset M C R". Before we proceed, we will need the 
following result. 

Lemma 4.4 (Path-connected lifts). Let M C R™ be a path- connected set and 
assume that for any permutation a G S, we either have o~M — M or o~M l~l M = 0. 
Then A _1 (M sym ) is a path- connected subset of S™ . 

Proo/. Let X X ,X % be in A^fAfy" 1 ), and set Xi = A(JQ) G M sym n R|, for 
i G {1,2}. It is standard to check that the sets X^ 1 (xi) are path-connected manifolds 
for i = 1,2. Consequently the matrices Xj and Diag(a;i) can be joined via a path 
lying in X^ 1 (xi). Thus in order to construct a path joining X\ to X2 and lying in 
A _1 (Af sym ) it would be sufficient to join x\ to X2 inside M sym . This in turn will 
follow immediately if both 17x1,17x2 belong in M for some a G S. To establish this, 
we will assume without loss of generality that x\ lies in M. In particular, we have 
M fl R> 7^ and we will establish the inclusion X2 G M. 



To this end, consider a permutation a G £ satisfying ^2 G crM n R" 



Oi 



immediate goal is to establish oM n M 7^ 0, and thus crAf = M thanks to our 
assumption. To this end, consider the point y G M satisfying xi — cry. If y lies in 
R", then we deduce y = X2 and we are done. Therefore, we can assume y ^ R™. 
We can then consider the decomposition a = o~k ■ ■ • a\ of the permutation a into 2- 
cycles o~i each of which permutes exactly two coordinates of y that are not in the right 
(decreasing) order. For the sake of brevity, we omit details of the construction of such 
a decomposition; besides, it is rather standard. We claim now o\M = M . To see this, 
suppose that o~\ permutes the i and j coordinates of y where yi < yj and i > j. Since 
x\ lies in R" and M is path-connected, there exists a point z G M satisfying z\ = Zj . 
Then <j\z = z, whence o~\M = M and o~\y G M. Applying the same argument to a\y 
and o~\M with the 2-cycle 172 we obtain o~io~\M = M and o~20~\V G M. By induction, 
oM = M. Thus X2 G M and the assertion follows. D 



Stratification duality for spectral lifts. Consider a symmetric, convex poly- 
hedral function /: R n — > R together with its induced stratification Af of dom/. 
Then with each polyhedron M G Af, we may associate the symmetric set M sym . We 
record some properties of such sets in the following lemma. 

Lemma 4.5 (Properties of Af). Consider a symmetric, convex polyhedral func- 
tion f: R" — > R and the induced stratification Af of dom f . Then the following are 
true. 
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(i) For any set Mi, M 2 E Af and any permutation a G E, the sets uM\ and M 2 

either coincide or are disjoint. 
(ii) The action of E on R ra induces an action of E on 

A) := {M E Af : dimM = k} 

for each k = 0, . . . ,n. in particular, the set M syra is simply the union of all 
polyhedra belonging to the orbit of M under this action. 
(iii) For any polyhedron M E Af, and every point x E M, there exists a neigh- 
borhood Uofx satisfying U Ci M sym = UnM. Consequently, M sym is a C°° 
manifold of the same dimension as M. 
Moreover, A _1 (Af sym ) is connected, whenever M is. 

The last assertion follows from Lemma l4.4l The remaining assertions are straight- 
forward and hence we omit their proof. 

Notice that the strata of the stratification Af are connected C°° manifolds, which 
fail to be symmetric in general. In light of Lemma |4~5I the set M sym is a C°° manifold 
and a disjoint union of open polyhedra. Thus the collection 

Af m := {M sym : M E Af}, 

is a stratification of dom /, whose strata are now symmetric manifolds. Even though 
the new strata are disconnected, they give rise to connected lifts A~ 1 (M sym ). One can 
easily verify that, as before, Jf restricts to an invertible mapping Jf : A s I m — > *4 s Z m 
with inverse given by the restriction of Jf . 

We now arrive at the main result of the section. 

Theorem 4.6 (Lift of the duality map). Consider a symmetric, convex polyhedral 
function f : R" — > R and define the spectral function F := f o A. Let Af be the finite 
polyhedral partition of dom / induced by f, and define the collection 



A, 



h-^M'y™) :MeA f \. 



Then the following properties hold: 

(i) Af is a C°° -stratification of dom F comprised of connected manifolds, 
(ii) F is C°°-partly smooth relative to each set A _1 (Af sym ) E Af- 
(iii) The assignment Jf ■ P(S") —} P(S") restricts to an invertible mapping Jf '■ At 

Af* with inverse given by the restriction of Jf* ■ 
(iv) The following diagram commutes: 

A jF A 

A f > A F * 



A s y m > ^ s y m 

/ Jf f * 

That is, the equation (A" 1 o J / )(A/ sym ) = (J F o A- 1 )(M sym ) holds for every 
set M sym E Aj m . 
Proof. In light of Lemma |4~51 each set M sym E A s ? m is a symmetric C°° manifold. 
The fact that Af is a C°°-stratification of domF now follows from the transfer princi- 
ple for stratifications [2J Theorem 4.8], while the fact that each manifold A -1 (M sym ) 
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is connected follows immediately from Lemma 14.51 Moreover, from Theorem l3.21l we 
deduce that F is C°°-partly smooth relative to each set in Af- 

Consider now a set M sym 6 _4 s / m for some M 6 Af. Then we have: 

J F (A^ 1 (M sym )) = |J ridF{X) 

U {U T (Dmgv)U : v£Tidf(X(X)) and U e O^}, 

and concurrently, 

\- 1 (J,(M* m ))=\- 1 ( |J iidf(xj)= |J O n .(Diagi;). 

xe Afsym xeM sy "\ v£ridf(x) 

We claim that the equality X' 1 {Jf {M s y m )) = ^(A^Af^ 111 )) holds. The in- 
clusion "d" is immediate. To see the converse, fix a point x £ M sym , a vector v G 
ridf(x), and a matrix U 6 O". We must show V := t/ T (Diag «)[/ £ l 7F(A -1 ( AfBym ))' 
To see this, fix a permutation a 6 £ with erx 6 R™ , and observe 

U T (Diagv)U = (A rT U) T (Diagav)A <T U, 

where A a denotes the matrix representing the permutation a. Define a matrix 
X := (A rT U) T (Biagax)A a U. Clearly, we have V S ri9F(X) and X g A _1 (M syin ). 
This proves the claimed equality. Consequently, we deduce that the assignment 
Jp\ P(S") — > P(S n ) restricts to a mapping Jp\ Af — > Af*, and that the diagram 
commutes. Commutativity of the diagram along with the fact that J7/* restricts to 
be the inverse of Jf : A s I m — > *4 s Z m implies that Jf* restricts to be the inverse of 
J F ■■ A F ->• Af* ■ □ 

Example 4.7 (Constant rank manifolds). Consider the closed convex cones of 
positive (respectively negative) semi-definite matrices S™ (respectively S"). Clearly, 
we have equality SJ = A _1 (R![.). Define the constant rank manifolds 

M^ ~ {X e SJ : rankX = k}, for k = 0, . . . , n. 

Then using Theorem 14.61 one can easily check that the manifolds M k and M^_ k are 
dual to each other under the conjugacy correspondence 5s n <— > ^s™ ■ 

5. Extensions to nonsymmetric matrices. Consider the space of n x to real 
matrices M nxm , endowed with the trace inner-product (X, Y) = ti(X T Y), and the 
corresponding Frobenius norm. We will let the group O™'™ := O™ x O m act on M raxm 
simply by defining 

([/, V).X = U T XV for all ([/, V) £ O n - m and X e M" xm . 

Recall that singular values of a matrix A 6 M™ xm are defined to be the square roots of 
the eigenvalues of the matrix A T A. The singular value mapping a : M" xm — > R m is 
simply the mapping taking each matrix X to its vector (a\{X), . . . , a m {X)) of singular 
values in non- increasing order. We will be interested in functions F : M nxm — > R 
that are invariant under the action of O n,m . Such functions F can necessarily be 
represented as a composition F = f o a, where the outer-function / : R m — >• R is 
absolutely permutation-invariant, meaning invariant under all signed permutations of 
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coordinates. As in the symmetric case, it is useful to localize this notion. Namely, we 
will say that a function / is locally absolutely permutation-invariant around a point 
x provided that for each signed permutation a fixing x, we have f(o~x) — f(x) for 
all x near x. Then essentially all of the results presented in the symmetric case have 
natural analogues in this setting (with nearly identical proofs). 

Theorem 5.1 (The nonsymmetric case: lifts of manifolds). Consider a matrix 
X G M nxm and a set M C R™ 1 that is locally absolutely permutation-invariant around 
x := a(X). Then M is a C°° manifold around x if and only if the set a (M) is a 
C°° manifold around X . 

Proposition 5.2 (The nonsymmetric case: lifts of identifiable sets). Consider 
a Isc f ': R m — > R and a matrix X G M nxm . Suppose that f is locally absolutely 
permutation-invariant around x := o~(X) and consider a subset M C R m that is 
locally absolutely permutation-invariant around x. Then M is identifiable (relative 
to f) at x for v 6 df{x), if and only if o~~ x (M) is identifiable (relative to f o a) 
at X for U 7 (Diagu)!^ G d(f o a)(X), where (U,V) G O n ' m is any pair satisfying 
X = U T (Dia,ga(X))V. 

Theorem 5.3 (The nonsymmetric case: lifts of partly smooth manifolds). Con- 
sider a Isc function f : R m — > R and a matrix X G M nxrn . Suppose that f is locally 
absolutely permutation-invariant around x := o~(X). Then f is C°° -partly smooth at 
x relative to M if and only if f o a is C°° -partly smooth at X relative to a~ 1 (M). 

It is unknown whether the analogue of the latter theorem holds in the case of 
C p partial smoothness, where p < oo. This is so because it is unknown whether 
a nonsymmetric analogue of [111 Theorem 4.21] holds in case of functions that are 
differentiable only finitely many times. 

Finally, we should note that Section |4] also has a natural analogue in the nonsym- 
metric setting. For the sake of brevity, we do not record it here. 

Acknowledgments. The first author thanks Nicolas Hadjisavvas for useful dis- 
cussions leading to a simplification of the proof of Lemma 14.41 

REFERENCES 

[1] F. Al-Khayyal and J. Kyparisis. Finite convergence of algorithms for nonlinear programs and 
variational inequalities. J. Optim. Theory Appl, 70(2):319-332, 1991. 

[2] E. Asplund. Differentiability of the metric projection in finite-dimensional Euclidean space. 
Proc. Amer. Math. Soc, 38:218-219, 1973. 

[3] J. Bolte, A. Daniilidis, and A.S. Lewis. Generic optimality conditions for semialgebraic convex 
programs. Math. Oper. Res., 36:55-70, 2011. 

[4] J.M. Borwein and A.S. Lewis. Convex analysis and nonlinear optimization. CMS Books in 

Mathematics/Ouvrages de Mathematiques de la SMC, 3. Springer- Verlag, New York, 2000. 

Theory and examples. 
[5] J.M. Borwein and Q.J. Zhu. Techniques of Variational Analysis. Springer- Verlag, New York, 

2005. 
[6] J.V. Burke. On the identification of active constraints. II. The nonconvex case. SIAM J. Numer. 

Anal, 27(4):1081-1103, 1990. 
[7] J.V. Burke and J.J. More. On the identification of active constraints. SIAM J. Numer. Anal., 

25(5):1197-1211, 1988. 
[8] P.H. Calamai and J.J. More. Projected gradient methods for linearly constrained problems. 

Math. Program., 39(1):93-116, 1987. 
[9] F.H. Clarke, Yu. Ledyaev, R.I. Stern, and P.R. Wolenski. Nonsmooth Analysis and Control 

Theory. Texts in Math. 178, Springer, New York, 1998. 



Orthogonal Invariance and Idcntifiability 21 

[10] A. Daniilidis, A.S. Lewis, J. Malick, and H. Sendov. Prox-regularity of spectral functions and 

spectral sets. J. Convex Anal., 15(3):547-560, 2008. 
[11] A. Daniilidis, J. Malick, and H.S. Sendov. Locally symmetric submanifolds lift to spectral 

manifolds, preprint U.A.B. 23/2009, 4-3 p., arXiv: 1212.3936 [math.OC]. 
[12] A. Daniilidis, C. Sagastizabal, and M. Solodov. Identifying structure of nonsmooth convex 

functions by the bundle technique. SIAM J. Optim., 20(2):820-840, 2009. 
[13] C. Davis. All convex invariant functions of hcrmitian matrices. Arch. Math., 8:276—278, 1957. 
[14] D. Drusvyatskiy and M. Larsson. Approximating functions on stratifiable sets. Under review, 

arXiv:1207.5258 [math.CA], 2012. 
[15] D. Drusvyatskiy and A.S. Lewis. Optimality, idcntifiability, and sensitivity. Under review, 

arXiv.1207. 6628 [math.OC]. 
[16] J.C. Dunn. On the convergence of projected gradient processes to singular critical points. J. 

Optim. Theory Appl, 55(2):203-216, 1987. 
[17] M.C. Ferris. Finite termination of the proximal point algorithm. Math. Program. Ser. A, 

50(3):359-366, 1991. 
[18] S. Fitzpatrick and R.R. Phelps. Differentiability of the metric projection in Hilbert space. 

Trans. Amer. Math. Soc., 270(2):483-501, 1982. 
[19] S.D. Flam. On finite convergence and constraint identification of subgradicnt projection meth- 
ods. Math. Program., 57:427-437, 1992. 
[20] W.L. Hare and A.S. Lewis. Identifying active constraints via partial smoothness and prox- 
regularity. J. Convex Anal., ll(2):251-266, 2004. 
[21] R.B. Holmes. Smoothness of certain metric projections on hilbert space. Trans. Amer. Math. 

Soc., 184:pp. 87-100, 1973. 
[22] A.S. Lewis. Derivatives of spectral functions. Math. Oper. Res., 21(3):576-588, 1996. 
[23] A.S. Lewis. Nonsmooth analysis of eigenvalues. Math. Program. Ser. A, 84(l):l-24, 1999. 
[24] A.S. Lewis. Active sets, nonsmoothness, and sensitivity. SIAM J. Optim., 13:702-725, 2002. 
[25] A.S. Lewis and H.S. Sendov. Nonsmooth analysis of singular values. I. Theory. Set-Valued 

Anal, 13(3):213-241, 2005. 
[26] B.S. Mordukhovich. Variational Analysis and Generalized Differentiation I: Basic Theory. 

Grundlehren der mathematischen Wisscnschaftcn, Vol 330, Springer, Berlin, 2006. 
[27] R.R. Phelps. Convex functions, monotone operators and differentiability, volume 1364 of Lec- 
ture Notes in Mathematics. Springer- Verlag, Berlin, 2nd edition, 1993. 
[28] R.A. Poliquin and R.T. Rockafcllar. Prox-regular functions in variational analysis. Trans. 

Amer. Math. Soc, 348:1805-1838, 1996. 
[29] R.A. Poliquin, R.T. Rockafcllar, and L. Thibault. Local differentiability of distance functions. 

Trans. Amer. Math. Soc, 352(ll):5231-5249, 2000. 
[30] R.T. Rockafcllar and R.J-B. Wets. Variational Analysis. Grundlehren der mathematischen 

Wissenschaften, Vol 317, Springer, Berlin, 1998. 
[31] H.S. Sendov. The higher-order derivatives of spectral functions. Linear Algebra Appl, 

424(1):240-281, 2007. 
[32] M. Silhavy. Differentiability properties of isotropic functions. Duke Math. J., 104(3) :367-373, 

2000. 
[33] J. Sylvester. On the differentiability of O(n) invariant functions of symmetric matrices. Duke 

Math. J., 52(2):475-483, 1985. 
[34] J. von Neumann. Some matrix inequalities and metrization of matrix-space. Tomck. Univ. 

Rev., 1:286-300, 1937. 
[35] S.J. Wright. Identifiable surfaces in constrained optimization. SIAM J. Control Optim., 

31:1063-1079, July 1993. 
[36] E.H. Zarantoncllo. Projections on convex sets in Hilbert space and spectral theory. I. Projections 

on convex sets. In Contributions to nonlinear functional analysis (Proc Sympos., Math. 

Res. Center, Univ. Wisconsin, Madison, Wis., 1971), pages 237-341. Academic Press, 

New York, 1971. 



